T 


AD-767  710 


OPTIMAL  ALLOCATIONS  IN  THE  CONSTRUCTION 
OF  k-OUT-OF-n  RELIABILITY  SYSTEMS 


Sheldon  M.  Ross,  et  al 


California  University 


Prepared  for: 


Office  of  Naval  Research 
Army  Research  Of  f  ice  -  Dur  ha  m 


September  1973 


DISTRIBUTED  BY: 


National  Technical  Information  Sonrico 
U.  S.  DEPARTMENT  OF  COMMERCE 

5285  Port  Royal  Road,  Springfield  Va.  22151 


■ -T.  jr  V 


- 1  ni  I  <ir~h  i  JrSt  fiWlfi  attr’ii'i'f'  I'l'iaiWiiAllfliii 


or  73-17 


OPTIMAL  ALLOCATIONS  IN  THE  CONSTRUCTION 
k-OUT-OF-n  RELIABILITY  SYSTEMS 


SHODON  M.  ROSS, 
CYRUS  HERMAN  and 
GERALD  j.  UEtERMAN 

iH 

CO 

o 

< 


OPERATIONS 

RESEARCH 

CENTER 


D  D  C\\ 

K'rpunprpra 


S  0CT  15  1975 


R*p,odul.-.n.,  fr , 

NATIONAL  TECHNICAL  J  Lb  L 
INFORMATION  SERVICE 

U  5  1  c«  Conimcrcn 

S|.-  .•'<!  VA  22151 


-pBTMBUTiON  STATEMENT  A 
Approve  lor  Yjwblic  rol«ca« 

DLRlribu'don  Unllmifd 


COLLEGE  OF  ENGINEERING 
UNIVERSITY  OF  CALIFORNIA  •  BERKELEY 


V 


J 


OPTIMAL  ALLOCATIONS  IN  THE  CONSTRUCTION  OF  k-OUT-OF-n  RELIABILITY  SYSTEMS 


by 


Sheldon  M.  Ross 

Department  of  Industrial  Engineering 
and  Operations  Research 
University  of  California,  Berkeley 

and 

Cyrus  Derman 
Civil  Engineering 
Division  of  Mathematical  Methods 
and  Operations  Research 
Columbia  University 
New  York,  New  York 

and 

Gerald  J.  Lieberman 
Operations  Research  Department 
Stanford  University 
Stanford,  California 


SEPTEMBER  1973  ORC  73-17 

T 

This  research  has  been  partially  supported  by  the  U.S.  Army  Research 
Of f ice-Durham  under  Contract  DA-31-124-ARO-D-33 1  and  the  Office  of 
Naval  Research  under  Contract  N00014-69-A-0200-1036  with  the  University 
of  California.  Reproduction  in  whole  or  in  part  is  permitted  for  any 
purpose  of  the  United  States  Government. 

i 

/ 


Unclassified 


Security  Classification  _ 

DOCUMENT  CONTROL  DATA  •  R  &  D 


i  J.issihvutton  ut  tttlv.  hn,l\  ut  ,t  h>tr*nt  mu!  tudvnmg  .ntnuMfinn  mn»f  in-  vntvrvd  ulu-n  thv  owraf/  report  is  classified) 


t  ORIGINATING  A  c  Tl  VI  T  V  f  Corpora  F*  imthut) 

NIPONT  SECURITY  CL  ASSIFICATION 

University  of  California,  Berkeley 

■ 

Unclassified 

2h.  CROUP 

i  u  •  p  o  n  t  r  i  t  l  r 


OPTIMAL  ALLOCATIONS  IN  THE  CONSTRUCTION  OF  k-OUT-OF-n  RELIABILITY  SYSTEMS 


4  DESCRIPTIVE  NOTES  (Type  of  report  *nd9  inclusive  da  ton) 


Research  Report 


s  au  TMORiSi  (First  name,  middle  initial,  last  name) 

Sheldon  M.  Ross,  Cyrus  Derman  and  Gerald  J.  Iioberman 

6  REPORT  DATE 

September  1973 

7 a.  TOTAL  NO.  OF  PAGES 

21 

7b.  NO  OF  REFS 

1 

«.».  CON  TRACT  OR  CHANT  NO. 

N00014-69-A-0200-1036 

b.  PROJEC  T  NO. 

NR  042  238 

C. 

Research  Project  No,:  WW  041 

d. 

9u.  ORIGINATOR'S  REPORT  NUMB  E  R  (S) 

0RC  73-17 

9b.  OTHER  REPORT  N  0 1 S 1  (Any  other  numbers  that  may  be  ms  signed 
this  report) 

10  DISTRIBUTION  STATEMENT 


This  document  has  been  approved  for  public  release  and  sale;  its  distribution  is 
unlimited. 


"  notes  Also  supported  by  the 

12.  SPONSORING  MILITARY  ACTIVITY 

U.S.  Army  Research  Of fice-Durham  under 

Office  of  Naval  Research 

Contract  DA-31-124-ARO-D-331. 

Department  of  the  Navy 

Arlington,  Virginia  22217 

"  3  ABSTRACT 

SEE  ABSTRACT. 

DD  ,"".“.,1473  (PACE  I) 

S/N  01 01 -807-681 1 


1 


Unclassified 

Security  Classification 


A- .1M0II 


Unclassified 

Security  Classification 


KCV  *OSO! 


Reliability 
k-Out-of-n 
Optimal  Policy 
Sequential  Case 
Allocating  Resources 
Dynamic  Programming 


,Fr\,1473  «back» 


ABSTRACT 


We  want  to  build  n  components  so  as  to  form  an  n  com¬ 
ponent  system  which  will  function  if  at  least  k  of  the 
components  function.  If  x  dollars  is  invested  in  building 
a  component,  then  this  component  will  function  with  prob¬ 
ability  P(x)  .  Given  a  total  income  of  A  dollars,  the 
problem  of  interest  is  to  determine  how  much  money  we  should 
invest  in  each  component  so  as  to  maximize  the  probability 
of  attaining  a  functioning  system.  This  problem  is  con¬ 
sidered  both  in  the  sequential  and  in  the  nonsequential  case. 
Conditions  under  which  it  is  optimal  to  allocate  A/n  units 
at  each  stage,  when  A  is  your  initial  fortune,  are 
presented.  The  special  case  P(x)  ■  min  (x,l)  is  also 
considered  in  detail. 


-  _ •  •'  >  '  --  - 


OPTIMAL  ALLOCATIONS  IN  THE  CONSTRUCTION  OF 
k-OUT-OF-n  RELIABILITY  SYSTEMS 


C.  Derman,  G.  J.  Lieberman,  S.  M.  Ross 


1.  Introduction. 

We  want  to  build  n  components  so  as  to  form  an  n  component 
system  which  will  function  if  at  least  k  of  the  components  function. 

If  x  dollars  is  invested  in  building  a  component  then  this  component 
will  function  with  probability  .P(x),  where  P(x)  is  an  increasing 
function  such  that  P(0)  =  0.  We  have  a  total  income  of  A  dollars. 

The  problem  of  interest  is  to  determine  how  much  money  we  should  invest 
in  each  component  so  as  to  maximize  the  probability  of  attaining  a 
functioning  system.  We  will  be  interested  in  this  problem  both  in  the 
sequential  and  in  the  nonsequential  case.  In  the  sequential  case  we 
assume  that  the  individual  components  are  built  sequentially  in  time  and 
that  knowledge  as  to  whether  or  not  a  component  functions  is  available 
to  us  before  we  have  to  allocate  our  investment  in  the  next  component. 

In  the  nonsequential  case  it  is  assumed  that  all  allocations  must  be 
simultaneously  made. 

In  Section  2  of  this  paper  we  consider  the  case  k  =  1  and 
present  conditions  on  P(x)  under  which  it  is  optimal  to  put  an  equal 
investment  in  all  n  components  and  conditions  under  which  it  is  optimal 
to  put  the  total  fortune  A  into  a  single  component.  In  Section  3,  it 


1 


is  shown  that  the  "equal  investment"  condition  carries  over  to  the  case 
of  general  k.  In  Section  k  we  consider  the  special  case  P(x)  =  x 
in  the  sequential  situation  and  determine  the  optimal  policy  when  k  =  2. 

A  conjecture  as  to  the  optimal  policy  in  the  general  case  is  also  made. 
Several  remarks  are  made  concerning  the  non- sequential  case  with  P(x)  =  x 
(considered  in  [l])  in  Section  5*  In  the  final  section  we  consider  a 
related  problem. 


2.  The  Case  k  =  1. 

r, 

When  k  =  1  the  sequential  and  nonsequential  cases  are  identical* 

both  are  involved  in  determining  x  =  (x^,  ...  ,  xr)  with  x^  >  0, 

n  n 

2  x.  =  A  so  as  to  maximize  [l  -  II  ( 1  -  P(x.))],  the  reliability  in  the 

1  1  i  1 

1  1  n  i- 1 

latter  case,  and  the  identical  expression  p(x.)  +2  H  (l-p(x  ))  p(xj) 

i=2  j=l  J 

in  the  former  case.  The  interpretation  of  x  is  that  x^^  dollars  is  to 

be  invested  in  component  i  in  the  nonsequential  case,  and,  in  the 

sequential  case,  dollars  is  to  be  invested  in  the  ith  attempt  if  the 

first  i-1  attempts  to  build  a  functioning  component  are  unsuccessful. 

n 

The  above  is  equivalent  to  choosing  x^,  ...  ,  xn,  x^  >  0,  2  x^  =  A 

n 

so  as  to  minimize  ^  l°g(  1  -  P(x.)). 

i=l 

Proposition  1; 

(a)  If  log(  1  -  P(x))  is  convex  then  the  optimal  allocation  is 

*1  =  X2  =  '*'  =  xn  =  A/n 
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(b)  If  log(  I  -  P(x))  is  concave  Chen  an  optimal  allocation  is 

*1  =  *,  x2  =  • • •  =  xn  =  0  . 

Proof:  Follows  from  standard  resulcs  about  concave  and  convex 
functions.  Q 

Remarks : 

( i)  In  part  (b)  the  condition  that  log(  1  -  P(x))  be  concave  can  be 
weakened  to  the  condition  that  log( 1  -  P(x))  be  subadditive, 
i.e.,  that  is,  (1  -  P(x  +  y))  <  (1  -  P(x))(l  -  P(  y) ) . 

f, 

( ii)  The  condition  that  log( 1  -  P(x))  be  superadditive,  i.e., 

(1  -  P(x  +  y))  >  (1  -  P(x))  (1  -  P(y)),  would  not  be  sufficient 
to  establish  part  (a).  It  would,  however,  necessarily  imply  that 
the  optimal  x  vector  would  have  all  positive  components. 

3.  The  General  Case. 

Part  (a)  of  Proposition  1  remains  true  in  the  general  case. 

Theorem  1:  If  log(  1  -  P(x))  is  (strictly)  convex  then  when  one  wants 
to  sequentially  build  k  working  components  in  at  most  n  attempts, 
n  >  k,  then  it  is  (uniquely)  optimal  to  allocate  A/n  at  each  stage 
when  A  is  your  total  resources. 


F 


«l ;« 'f.ij  '*■«  i,.W« 


if-  f 

I 

I  1 


Proof;  Assume  first  that  log( 1  -  P(x))  is  strictly  convex.  The  proof 

is  by  induction  on  k,  and,  as  we  have  already  proven  the  result  when 

k  =  1  let  us  assume  that  it  is  true  for  all  values  less  than  k.  Now 

consider  the  k  component  case.  If  the  number  of  possible  stages  is 

k  then  a  policy  reduces  to  a  vector  ...  ,  x^)  where  x^  >  0, 

k 

£  x.  <  A,  with  the  interpretation  that  the  policy  invests  x^  in  the 
1  i 

first  stage  and  if  the  first  i-1  stages  all  result  in  working  components 
then  x^  is  invested  in  the  ith  stage.  (That  is,  when  the  number  of 
available  stages  is  identical  with  the  number  of  desired  components  then 
the  sequential  problem  reduces  to  the  nonsequential  one.)  Thus,  the 

t, 

problem  reduces  to 

k  k 

Maximizing  II  P(x.)  subject  to  x  >  0,  £  x  =  A 
i=l  1  1  1  1 

or,  equivalently,  to, 

k  k 

Maximizing  2  lo8  p(x  )  subject  to  x  >  0,  2  x<  =  A- 
1  1  1  "  1 

New  the  strict  convexity  of  log( 1  -  P(x))  implies  that  log(P(x))  is 
strictly  concave  (see  the  following  Lemma  1).  Hence,  by  standard 
arguments,  it  follows  that  x^^  =  A/k,  i  =  1,  ...  ,  k  is  the  (unique) 
optimal  allocation  and  the  result  is  established  in  this  case. 

Thus  the  result  is  true  in  the  k  component  case  when  the 
number  of  stages  is  also  equal  to  k.  So  let  us  assume  that  the  result 
holds  in  the  k  component  case  whenever  the  number  of  stages  is  less 
l!>an  n  and  try  to  prove  that  it  also  holds  when  the  number  of  stages 


iiiiiifiiteiute. 


. . .  ■ 


—■in  r  il 


equals  n.  To  do  so  suppose  that  an  optimal  policy  for  a  k-component 
n-stage  problem  initially  allocates  an  amount  x^.  Now  if  this  initial 
attempt  is  successful  then  the  induction  hypothesis  (on  the  number  of 
desired  components)  tells  us  that  it  is  uniquely  optimal  from  that  point 
on  to  allocate  (A-x^)/(n-l)  for  each  of  the  remaining  n-1  stages.  On 
the  other  hand  even  if  the  initial  attempt  is  unsuccessful  then  as  there 
are  only  n-1  stages  to  go  it  follows  by  the  induction  hypothesis  on  the 
number  of  stages  in  the  k-component  case  that  it  is  still  optimal  to 
allocate  (A-x^)/(n-l)  on  each  of  the  remaining  n-1  stages.  If 
Xj  -  A/n  then  the  result  is  proven.  So  let  us  suppose  that  x^  £  A/n 
and  obtain  a  contradiction.  If  x^  ^  A/n  then  consider  the  policy  that 
allocates  ^  [x^  +  (A-x^)/(n-l) ]  for  each  of  the  first  two  stages  and 
then  allocates  (A-x^)/(n-l)  for  each  of  the  last  n-2  stages.  This 
policy  is  thus  identical  with  the  optimal  policy  during  the  last  n-g 
stages.  Now  the  probability  that  at  least  one  of  the  components  built 
during  the  first  two  stages  is  successful  is 

(1)  1  -  [1  -  Pfx^l  [l  -  P  (^) 


under  the  optimal  policy;  while  it  is 


It  is  easy  to  show  that  an  optimal  policy  exists  for  the  n  stage 
problem.  This  is  done  by  first  proving  recursively  that  the  optimal 
value  function  for  an  n  stage  problem  is  a  continuous  function  of  the 
initial  fortune  whenever  P(x)  is  continuous.  An  optimal  policy  then 
exists  since  a  continuous  function  obtains  its  maximum  on  a  closed  set. 
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under  Che  new  policy.  Also  the  probability,  that  both  components  are 
successful  is 


A-x 

(3)  P(xt)  P  ^ 

for  the  optimal  policy*  while  it  is 


i  r  A-*i]  \ 


for  the  new  policy.  It  follows  by  the  results  given  for  k  =  1  that 

(2)  is  greater  than  (1)  and  it  follows  from  that  fact  that  log(P(x)) 

'  r, 

is  concave  (Lemma  1)  that  (4)  is  greater  than  (3).  Hence,  under  the  new 
policy  the  number  of  successes  during  the  first  two  stages  is  stochastically 
greater  than  it  is  under  the  optimal  policy.  As  the  two  policies  are 
identical  after  the  first  two  stages  it  thus  follows  that  the  probability 
of  at  least  k  successes  is  greater  under  the  new  policy  than  it  is 
under  the  optimal  policy.  This  contradiction  shows  that  x^  a  A/n, 
which  proves  the  result  in  the  k-component  case  for  any  number  of  stages, 
which  also  completes  the  initial  induction  proof. 

If  log( 1  -  P(x))  is  convex  but  not  strictly  so,  then  we  can 
approximate  log( 1  -  P(x))  arbitrarily  closely  by  functions  that  are 
strictly  convex  and  then  apply  a  continuity  argument.  | 


The  following  lemma  was  used  in  the  proof  of  the  theorem. 


Lemma  1 :  If  0  <  P(x)  <  1,  and  Log( 1  -  P(x))  is  convex  then 
log(P(x))  is  concave. 

Proof:  Suppose  the  hypothesis  of  the  lemma  are  true.  Then 


“4 


1  og(  1  -  P(x)  )  = 


.  Pfx)  P^x)  -  (P'(x))2  -  P»(x) 


(1  -  P(x))‘ 


implying  that 


P"(x) 


< 


1  -  P(x) 


<  0 


Now, 


-l  log  rCx)  m  P(x)  P"(x)._-  (P’(x))g 


dx£ 


(*(*)} 


which  is  negative  since  P"(x)  <  0. 


Remarks. 

(  i)  It  follows  from  Theorem  1  that  when  log(l  -  P(x))  is  convex  the 
optimal  sequential  policy  is  nonsequential  is  nature,  and  is 
thus  also  the  optimal  policy  when  the  allocations  for  each  stage 
must  be  made  simultaneously  rather  than  sequentially. 

(ii)  If  we  think  of  P(x)  as  being  a  probability  distribution  function 
then  the  condition  that  log( 1  -  P(x))  be  convex  is  equivalent  to 
the  condition  that  P(x)  is  a  decreasing  failure  rate  distribution. 
Since  mixtures  of  decreasing  failure  rate  distributions  are  them¬ 
selves  decreasing  failure  rate  distributions  it  thus  follows  that 
log( 1  -  P(x) )  will  be  convex  whenever 
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where 


and 


P(x)  =  /  P  (x)  dF(a) 

a 


log(  1  -  P^(x))  is  convex  for  all  a, 


F (Ct)  is  a  probability  distribution  function. 
In  particular,  any  P(x)  of  the  form 

P(x)  =  /  (1  -  e’0*)  dF(a) 
a 


will  be  such  that  log( 1  -  P(x))  is  convex. 

( iii)  If  we  let  V^(A)  denote  the  probability  that  at  least  k  successes 
will  occur  in  the  n  stages  under  an  optimal  policy  then  from 
Theorem  1  it  follows  that 

V  ( A)  =  2  (")  ( P(  A/n) ) 1  (1  -  P( A/n) ) n“ 1 
n  i=k  1 


If  P(x)  is  differentiable  then 

lim  =  lim  P’(x)  =  P '( 0)  s  X; 

x  ->0  x  — *  0 

and  thus  from  the  Poisson  approximation  to  the  binomial  distribution 
it  follows  that 

00 

V  (A)  t  e  ^  2  (AA)*Vk.’  as  n  t  » 

n  i=k 

(This  is  so  since  Vr(A)  =  Prob(Bin(n,  P(A/n))  >  k  where 
Bin(n,  P(A/n))  represents  a  binomial  random  variable  with 
parameters  n  and  P(A/n)  .)  It  should  be  pointed  out  that  this 
convergence  is  not  necessarily  monotone  for  an  arbitrary  differ¬ 
entiable  function  P(x)  with  P(0)  =  0. 
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X. 


While  the  analogue  of  part  (a)  of  Proposition  1  remains  true 
in  the  case  of  arbitrary  k  it  is  obvious  that  the  same  cannot  be  said 
of  part  (b).  In  this  section  we  consider  the  special  case  P(x)  =  x. 

Suppose  that  we  need  to  sequentially  build  two  (k  =  2)  function¬ 
ing  components  in  at  most  n  attempts  when  P(  x)  =  min(x,l).  Suppose 
that  our  initial  fortune  is  A  and  consider  the  policy  v  which  sequentially 
allocates  A/n  at  each  stage  until  a  functioning  component  is  built>  and. 


The  next  proposition  states  that  Un(y)  satisfies  what,  in 
dynamic  programming  terminology,  is  known  as  the  optimality  equation. 


Proposition  3'  For  y  <  1 

U  (y)  =  Max  [x(y-x)  +  { 1-x)  Unl(y-x)] 
0  <  x  <  y 


Proof:  Define 


f(>.)  =  x(y-x)  +  (1-x)  Uftl(y-x) 


=  x(y-x)  +  (1-x 
Differentiation  yields 


n.n-1 

‘•a 

V 


+  y  -  x 


£'(x)  =  (1-x)  ll  -  £* 


n-2  f  tn-1 


implying  that  f'(x)  =  0  if  and  only  if 


1  -  x  =  1  - 

n-1 


or 


x  = 


_  1 


Since  f"(y/n)  =  0,  it  follows  that  f(x)  attains  its  maximum  value  at 
x  =  y/n.  This  proves  the  result  since  f(y/n)  =  Un(y).B 


Theorem  2:  For  an  initial  fortune  of  A  <  1  the  policy  ir  maximizes  the 
probability  of  obtaining  two  functioning  components. 
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Proof:  From  Proposition  3  we  see  that 


(5)  Un(y)  >  x(y-x)  +  (  1-x)  Unl(y-x)  for  all  0<x<y 

The  right  side  of  the  above  can  be  interpreted  as  the  return  (i.e., 
probability  of  obtaining  two  functioning  components)  if  x  is  allocated 
for  the  first  stage  and  then  policy  v  is  used  for  the  remaining  stages. 

As  the  inequality  (5)  holds  for  all  x,  0  <  y  <  x(  and,  as  we  already 
know  that  77-  is  optimal  when  only  one  additional  functioning  com*  onent 
is  needed  (Proposition  1),  we  can  interpret  (5)  as  stating  that  using 
policy  7 r  is  better  than  doing,  anything  else  for  one  stage  and  then 
switching  to  tt.  Repeating  this  argument  yields  that  using  tt  is  better 
than  doing  anything  else  for  two  stages  and  then  switching  to  tt.  Finally, 
by  repeating  the  same  argument  a  total  of  n  times  we  .ee  that  using 
policy  7 r  is  better  than  doing  anything  else  for  the  first  n  stages. 

This  completes  the  proof.  "§ 


Remark: 

Since  Theorem  2  states  that  U  (y)  is  the  maximal  probability  when 

n 

at  most  n  stages  are  available  it  follows  that  U^(y)  is  an  increasing 


function  of  n  and  thus,  for  0  <  y  <  1, 


II 

U  (y)  =  (!-■£)  +y-ltey+y-l  as  n  t  « 


(This  constitutes  another  proof  that  the  convergence  of  ( 1  -  to 

“V 

e  7  is  monotone  when  0  <  y  <  i)  In  fact,  for  any  positive  y  it  it  well 
known  and  can  be  easily  shown  that  the  convergence  is  monotone  for  all 


n  >  N  where  y/N  <  1. 


The  policy  v  is  no  longer  optimal  when  our  initial  fortune 
can  be  greater  than  1.  Let  us  define  the  policy  t r*  to  be  such  that 
when  the  present  fortune  is  y  and  at  most  n  additional  components 
can  be  built. 

(a)  allocates  y  if  only  one  additional  working  component  is  needed; 
and 

(b)  if  two  additional  working  components  are  needed,  allocates 

n 


y  -  1 

I 

n 


if  r*n-l 


.  *  n 

lf 


If  we  let  V  (y)  denote  tne  probability  of  a  success  (two  working 
n 

components)  when  our  initial  fortune  is  y  and  policy  tt*  is  employed 

y  n- 1 

then,  as  y  <  n/(n-l)  implies  that  y  -  ^  ,  it  follows  that,  for 

0  <  y  <  2 


v„W  - 


u  (y) 
n'  ' 


lf  y<-7 
—  n-1 

y  -  i  +  (s-y)  un.i(1)  if  y 


or,  equivalently,  for  0  <  y  <  2 


v„(y)  - 


( 1  -  n>  +  y  -  1  ££  *<-£i 

(?-y)  <£!>"  l*y  -  £  ^  y>^rr 


Theorem  3 •  Policy  i t*  is  optimal. 

Proof:  The  proof  would  follow  exactly  as  in  the  proof  of  Theorem  2, 
if  we  could  show  that 
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(6) 

VR(y)  >  x  Min(y-x,  1)  +  (1-x)  Vnl(y-x) 

for  all  x  <Min(l,y) 

When 

y  <  i,  (6) 

is  identical  to  (5)  an d  thus  we 

only  need  prove  (6) 

when 

1  <  y  <  2. 

Hence  we  must  show  mat,  for 

1  <  y  <  2, 

(7) 

V  (y)  >  Max 
nw/  — 

j  Max  (x  +  (1-x)  V  (y-x 

L°  <  x  <  y-1  n_1 

)), 

Max  (x(y-x)  +  (1-x) 

y-1  <  x  <  1 

* 

1 

« 

> 

We  consider  2  cases. 


Case  1.  y  <  . 

Now, 

Max  (x  +  (1-x)  V  ,(y-x)] 

0  <  x  <  y- 1 

=  Max  (x  +  (1-x)  [(1  -  ~y)  +  y  -  x  -  1]) 

0  <  x  <  y-1 


Define  the  function  f(x)  by 


n- 1 


f(x)  s  x  +  (  1-x)  [(1  -  j£2)  +  y 


1] 


Now, 


and 


f'(x)  =  1  +  ( 1-x)  C(  1  -  ^Y)n'2-  1]  -  (1  -  ^)n_1  -  y  +  x  ,  1 
f"(x)  =  (1-x)  (^f)  (1  -  -  (1  -  ^)n'2  +  2  -  (1  -  £r)n'2>  0 
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I 


i 


3 

g 


i 


Hence  f(x)  is  a  convex  function  in  the  region  0  <  x  <  y-1,  and  thus 
obtains  its  maximum  value  in  this  region  either  at  x  =  0  or  x  =  y-l. 
Now 

f(°)  =  U  -  +  y  -  1 

and 

f(y-i)  =  y  -  i  +  (2-y) 

Now  the  function 

g(y)  =  f(y-i)  -  vjy)  =  (2-y)  1  -  (i  -  i)n 

is  zero  at  y  =  — ~  .  Differentiation  shows  that  it  is  an  increasing 

function  when  y  <  — and  thus  it  follows  that 
—  n-1 

f(y-l)  v  y  -  1  +  (1  -  y  =  Vn(y) 

Y  n 

As  was  previously  noted  (  1  -  is  increasing  in  n,  it  follows  that 

f(°)  <vn(y);  thus» 

Max  f(x)  <_  V  (y)  when  y  <  — 

0  <  x  <  y-l  n  n_i 

Also,  since  when  y  <  f  the  inequality 

Vn(y)  ^  Max  (x( y-x)  +  (1-x)  V  ,(y-x) ) 
y-i<x<l  n'L 

is  identical  to  the  inequality  (5),  it  follows  that  (7)  is  established 
when  y  <  .  We  are  thus  ready  for 
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•*wr 


Thus 


x  =  0  or  at  x  =  y  - 


n-2  ‘ 


g(x)  =  Max[g(0),  g(y  -  ■“ jj)] 


0  <  x  <  y 


Max 

n-1  ^  .  , 

y  -  — o  <  x  <  y-1 
7  n-2  —  — 


[x  +( 1-x) [( 1 


n~  \  «  <  V- 1 

n-2  -  X  -  y 


fri)n  l+  y-x-H) 


Now  it  has  previously  been  shown  that  the  above  function  in  brackets  is 
a  convex  function  and  thus 


(9)  Max  g(x)  =  Max[g(y  -  ^) ,  g(y-l)] 

n-1  ^  . 

Hence,  from  (8)  and  (9)  we  see  that  when  y  > 


( 10)  Max  g(  x) 
0  <  x  <  y-1 


n-^  n-2 

(2-y)  (£§)  +  y  -  i 

„  n-1  r2n-3  i  1  i 

Max  y  -  ^2  +  -  y]  [fe>)  + 


,  ,n\  /n-2Nn-l 

y  -  1  +  (2-y)  (tjty) 


On  the  other  hand,  when  — r  >  y  >  — r  .  we  obtain  that 

’  n-2  -  n- l  ' 


Max  g(x)  =  Max 


tl" w 

(2-y)  (ttio)  +  y  -  1 


°-’t-y'1  (2-y)  (^)n'1  *  y  -  1 


With  the  help  of  a  little  algebra  we  obtain  from  (10)  and  (11)  that 


when  y  >  — r 
—  n- 1 


Max  g(x)  =  y  -  1  +  (2-y)  (f=f)“  =  Vn(y) 

x  <  y-1 


and  the  proof  is  complete. 

Hence  the  optimal  policy  is  determined  in  the  case  k  =  2. 

If  n  =  eof  it  can  be  seen  that  no  optimal  policy  exists.  The  dynamic 
programming  functional  equation  will  have  a  solution;  however^  the 
solution  is  not  the  return  function  of  any  policy.  The  policy  determined 
by  the  solution  is  the  non-optimal  policy  of  allocating  x  =  0  at 
every  stage. 

In  the  general  case  consider  the  policy  t*  which  is  such  that 
if  our  present  fortune  is  y  and  if  k  additional  working  components 
are  needed  with  at  most  n  stages  to  go,  then  tt*  calls  for  allocating 


iJ 


y  -  (k-L)  if  y  >  “ fj  (k-l) 


(Note  that  tt*  corresponds  to  the  previously  defined  v*  when  k  =  2.) 

Define  V  ,(y)  to  be  the  probability  of  success  under  ir*  when  our 
n,  k 

present  fortune  is  y  and  k  additional  working  components  are  needed 
and  at  most  n  additional  components  can  be  built.  If  we  can  show 


(*>  Vnjk(y)> 


„  <  ,  ,  1v[*Vn-l,k-l<!,-x'  *  (1-x>  Vl,k<yx» 

0  <  x  <  min(y,  1)  '  * 
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fa*  ■fiat  / 


3  ipTTi gga 


then  it  would  follow  that  t*  is  the  optimal  strategy.  We  have,  at 
present,  been  unable  to  verify  ( 12)  but  we  conjecture  that  it  is  valid 
and  that  7 r#  is  optimal. 

A  simple  formula  for  ^(y)  in  the  case  y  <  1  is  obtained 
by  conditioning  on  the  number  of  steps  required  to  obtain  k-1  success¬ 
ful  component...  This  yields 


n* 1  r_i  v  k-1  v  r-k+1  rv 

*.kW*  ?  ,0$  <1‘  n>  <»-■?>. 

*  r=k-l 


5.  The  Non-Sequential  Case;  P(x)  =  x. 

The  non-sequential  case  with  P(x)  =  x  and  general  k  was 

considered  in  [l]  where  it  was  shown  that  the  optimal  x  vector 

x*-  =  (x* . x*)  is  such  that  all  of  the  non-zero  element  of  x* 

—  '  1  *  ’  n  — 

are  equal.  A  problem  of  interest  is  to  determine  for  0  <  y  <  k  that 
value  of  r  that  maximizes 


Q(  r)  =  P^{at  least  k  components  work) 


■  k  (5) 


for  r  =  k,  k+1,  ...  ,  n.  From  Proposition  1(b),  r  =  1  when  k  =  1. 

In  general  the  optimal  value  of  r  is  a  function  of  y  and  k  but 
at  present  only  seems  obtainable  by  numerical  methods.  However,  we 
can  make  several  remarks: 

( 1)  If  y  is  near  enough  to  k  then  the  optimal  value  of  r  is 

r  =  k.  To  see  this  note  that  when  y  =  k,  Q(k)  =  1  and  Q(r)  <  1 


if  r  >  k.  Since  Q(r)  is  a  continuous  function  of  y  for  each 
r  the  substance  of  the  remark  follows. 


(2)  For  every  r  there  exists  an  >  0  such  that  tor  y  <  e 

Q(r+1)  >  Q(r)  with  strict  inequality  holding  if  k  >  1.  Thus, 
loosely  speaking,  for  small  y  the  amount  of  redundancy  in  the 
optimal  allocation  is  large.  We  show  this  by  considering  for 
r  >  k 


lim 
y  ->  0 


Q(r) 

Q(  r+l) 


=  ( 1  +  r)  f1  '  rTl) 


=  R(k)  (say). 

We  need  only  show  that  R(k)  <  1  with  strict  inequality  holding  for 
k  >  1.  Now  R(  1)  =  1,  R(2)  =  1  -  1/r^  <  1.  However,  as  long  as 


r  >  k+1. 


R(  k+ 1) 
R(k)  = 


(1 


i  h±l 

r+l  *  r+l 
■  *  1  ■  • 


r+l  #  r-k 
r  *  r+l-k 


<  1  . 


Then,  R( k)  <1  for  all  k  >  1  and  the  proof  of  the  remark  follows. 
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6.  A  Related  Problem. 

Rather  than  maximizing  the  probability  of  sequentially  building  k 
successful  components  within  a  budget  of  A  we  shall  now  assume  that 
our  budget  is  unlimited  and  that  the  problem  is  to  minimize  the  expected 
amount  of  money  spent  in  obtaining  the  k  successful  components. 

Consider  first  the  case  k  =  1.  If  x  is  initially  allocated  and 
the  component  built  is  not  successful  (which  will  occur  with  probability 
1  -  P( x) )  then  the  situation  will  be  exactly  the  same  as  it  was  before 
the  initial  investment.  Hence,  if  it  was  initially  optimal  to  allocate 
x  then  it  will  still  be  optimal  to  allocate  x  for  the  second  component. 
Hence  a  policy  corresponds  'to  a  value  x  (in  dynamic  programming  termi¬ 
nology  we  are  restricting  attention  to  stationary  policies),  and  the 
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